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The Airspace Operations Laboratory at NASA Ames conducts research to provide a 
better understanding of roles, responsibilities, and requirements for human operators and 
automation in future air traffic management (ATM) systems. The research encompasses 
developing, evaluating, and integrating operational concepts and technologies for near-, 
mid-, and far-term air traffic operations. Current research threads include efficient arrival 
operations, function allocation in separation assurance and efficient airspace and trajectory 
management. The AOL has developed powerful air traffic simulation capabilities, most 
notably the Multi Aircraft Control System (MACS) that is used for many air traffic control 
simulations at NASA and its partners in government, academia and industry. Several 
additional NASA technologies have been integrated with the AOL ? s primary simulation 
capabilities where appropriate. Using this environment, large and small-scale system-level 
evaluations can be conducted to help make near-term improvements and transition NASA 
technologies to the FAA, such as the technologies developed under NASA’s Air Traffic 
Management Demonstration-1 (ATD-1). The AOL’s rapid prototyping and flexible 
simulation capabilities have proven a highly effective environment to progress the initiation 
of trajectory-based operations and support the mid-term implementation of NextGen. 
Fundamental questions about accuracy requirements have been investigated as well as real- 
world problems on how to improve operations in some of the most complex airspaces in the 
US. This includes using advanced trajectory-based operations and prototype tools for 
coordinating arrivals to converging runways at Newark airport and coordinating departures 
and arrivals in the San Francisco and the New York metro areas. Looking beyond NextGen, 
the AOL has started exploring hybrid human/automation control strategies as well as highly 
autonomous operations in the air traffic control domain. Initial results indicate improved 
capacity, low operator workload, good situation awareness and acceptability for controllers 
teaming with autonomous air traffic systems. While much research and development needs 
to be conducted to make such concepts a reality, these approaches have the potential to truly 
transform the airspace system towards increased mobility, safe and efficient growth in 
global operations and enabling many of the new vehicles and operations that are expected 
over the next decades. This paper describes how the AOL currently contributes to the 
ongoing air transportation transformation. 
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National Aeronautics and Space Administration 

Optimized Profile Descent 

Performance-Based Navigation 

Next Generation Air Transportation System 

Trajectory-Based Operations 

Time-Based Flow Management 

Traffic Management Advisor 

Terminal RADAR Approach Control 

Technology Readiness Level 

Separation Assurance 

Standard Terminal Automation Replacement System (TRACON Controller Workstation in the NAS) 
Terminal Automation Modernization and Replacement 
Voice Switching and Communication System 


I. Introduction 

F OR more than 1 5 years the Airspace Operations Laboratory (AOL) at NASA Ames has conducted research to 
provide a better understanding of roles, responsibilities, and requirements for human operators and automation 
in future air traffic management (ATM) systems. The research encompasses developing, evaluating, and integrating 
operational concepts and technologies for near-, mid-, and far-term air traffic operations. At the AIAA Modeling and 
Simulation Technologies conferences in 2006 and 2010 we presented overviews of the AOL’s capabilities and 
research with a focus on capabilities and features that were new at that time lj 2 . Following the theme of the 
preceding papers, this paper starts with a brief introduction of the main challenges and activities, and then describes 
the current laboratory capabilities with a focus on the most recent improvements. The second part of the paper 
provides an overview of the current research activities. 

NASA’s aeronautics research is aimed at solving the challenges that still exist in our nation’s air transportation 
system: air traffic congestion, safety and environmental impacts 2 3 . NASA’s Airspace Systems Program performs 
foundational research to enable the development of revolutionary improvements to, and modernization of, the 
National Airspace System (NAS). The AOL conducts research in support of both, the gradual modernization of the 
NAS through a thorough simulation evaluation of near-term improvements, as well as the development of 
revolutionary concepts. Therefore, the AOL engages not only in the integration and evaluation of high Technology- 
Readiness-Level (TRL) components in a high-fidelity lab environment, but also in the rapid prototyping of highly 
advanced potential future capabilities. Over the past few years the AOL, together with other major NASA ATM 
laboratories, participated in several research threads that required additional ground automation and flight-deck 
capabilities developed at NASA Ames’ Aviation System Division and at NASA Langley’s Air Traffic Operations 
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Laboratory (ATOL). A number of development activities in recent years were related to integration of these 
technologies. These efforts led to highly capable simulation platforms across several NASA laboratories. 

In addition to integrating other technologies, the AOL also added many new capabilities to its simulation 
platform, the Multi Aircraft Control System MACS 3 4 . These include more accurate emulations of the latest Terminal 
Radar Approach Control (TRACON) and en route controller workstation, many advanced scenario editing functions, 
increased support for trajectory based operations from take-off to touchdown, new controller tool prototypes, 
additional options for configuring system uncertainties, and increased support for autonomous operations. MACS 
has been distributed to many partners in government, industry and academia. Several groups have modified it further 
to meet their needs, for example to support research on Unmanned Aerial Systems or single pilot operations. MACS 
is being further improved and geared towards new vehicles and operations, keeping pace with current and future 
research needs. Except where noted all displays and functions described in the following section are part of MACS 
and its communication process ADRS. An overview over some of the current capabilities in the AOL is given next. 

II. Airspace Operations Laboratory Capabilities 


A. Laboratory Layout 

The AOL is located on the second floor of building N-262 at NASA Ames Research Center. The offices of the 
AOL’s research and development team are located directly next to the lab space. 

The laboratory extends 




across two areas that are in 
close proximity to each other. 

The “280” area is depicted in 
Figure 1 and the H211 area is 
depicted in Figure 2. 

There are eight workstations 
in room 280A and H209 that 
can be used by the 
experimenters to start and 
monitor the simulations. 60 
workstations are for the use of 
simulation participants, 

including air traffic 
controllers, air traffic 
managers, area supervisors, 
multi aircraft pilots and single 
aircraft pilots. The lab is laid 
out for maximum flexibility 
in conducting the research 
activities. All positions can be 
combined in one large simulation or many small 
simulations that do not interfere with each other. 
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Figure 1: AOL 280 area. 

“worlds” can be run in parallel, operating as independent 
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B. Air Traffic Control 

There are a total of 23 air traffic control sector positions in four air traffic control rooms (rooms 280E, 280F, H208 
and H209). Each room is equipped with a supervisor station, overhead projector and either five or six sector 
positions. Figures 3 and 4 show the 280E air traffic control room configured for TRACON operations during a 
simulation of ATD-1. 

The pictures show the wall projections of a MACS -based 
Traffic Situation Display (TSD) on the right wall, and a Traffic 
Management Advisor (TMA) 4 5 timeline display on the back 
wall. The sector controller positions are equipped with MACS- 
based emulations of the Standard Terminal Automation 
Replacement System (STARS). STARS is in operational use 
in many TRACONS in the US and will be phased into all 
TRACONS under the Terminal Automation Modernization 
and Replacement (TAMR) program 6 . Controllers use the 
STARS keyboards for data entry and conduct air/ground and 
ground/ground communications via a tablet, which hosts 
software developed by Quintron Systems, that emulates the 
fielded Voice Communication and Switching System (VSCS). 
Figure 4 also shows the supervisor station on the left that 
provides additional access to TMA and traffic data and is used 
to drive the overhead displays. 


Figure 3: Air Traffic Control room (280E) in 
TRACON configuration. 

Unlike the TRACON workstations, the en route 
controller workstations are available in two different look and feel settings. 

One configuration emulates the Display System Replacement (DSR) that 
was in operational use in the NAS for the past two decades. New since 
2013, MACS can also emulate the look and feel of the En Route 
Automation Modernization (ERAM) 7 radar workstations that are replacing 
the DSR in the NAS. Figure 5 shows an en route sector controller position 
with the ERAM keyboard and the MACS emulation of the radar display. 

The ERAM emulation in the AOL currently has several of the basic 
functions and menus, such as the macros, flight plan readouts, the 
continuous range readouts, the meter lists, and supports ERAM -specific 
processing of most of the command line inputs. Additional radar controller 
(R-side) views and functions, as well as various displays that are located 
on the ERAM side panel and/or radar associate (D-side) position, are 
currently under development in MACS. 

In addition to the sector controller positions shown here, MACS ATC 
stations can also be configured for traffic management positions and 
oceanic controller stations. MACS also includes the capability to ‘link’ 

ATC workstations, such that they can be used as R- and D-Side pairs that 
link display information between each other, such as data tag positions and 
contents, route displays and J-Rings. workstation with ERAM emulation. 



Figure 5: En route controller 
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C. Multi-Aircraft Flight Decks 

The multi-aircraft flight 
decks are used to control 
the majority of aircraft in a 
simulation. Generally one 
such pilot station is used to 
control the aircraft that are 
on a given controllers 
frequency. When the 
controller instructs the pilot 
to contact the next sector 
on a different frequency, 
the pilot selects this new 
frequency, which transfers 
the aircraft to the multi- 
aircraft station for that next 
sector. Figure 6 shows 
typical multi-aircraft 

workstations in the AOL’s 
H2 1 1 area. 

Figure 7 highlights some of the details encapsulated in the pilot stations. Aircraft flight IDs are shown in 
different lists. One list shows all flight IDs that are controlled by a given station. Prompts remind pilots to take 

specific actions when necessary. 
Some of the recent work in 
MACS has added additional 
filtering options for these 
reminders, enabling 

experimenters to customize them 
based on different criteria, such as 
route of flight, altitude, flight 
rules, equipage and more. 

When the flight ID is selected, 
the displays show the view from 
that particular aircraft and the 
pilot can use the input devices to 
control that aircraft’s flight 
management computer and flight 
control system. 

MACS supports using 
different aero models. The AOL 
and most other labs primarily use 
an enhanced point/mass model 
that is designed for ground- 
focused air traffic management 
research purposes. Other laboratories prefer a four-degree-of- freedom (4DOF) model that adds more aerodynamic 
characteristics required for flight deck-centered research. Each model supports the performance characteristics of all 
major aircraft types. Scale factors can be used on a per aircraft basis to simulate variations in climb/descent ratios of 
individual aircraft. 

D. Single-Aircraft Flight Decks: NASA Langley’s ASTOR System 

Several research projects require advanced flight deck operations and equipment, and/or validation of procedures 
and phraseology that cannot be easily conducted with MACS’ multi- or single -aircraft flight decks. Therefore, 
NASA Langley’s Aircraft Simulation for Traffic Operations Research (ASTOR) 8 has been fully integrated with the 
simulation capabilities in the AOL (see Figure 8). This enables the AOL, as well as other labs at NASA Ames, to 
include several ASTOR desktop flight simulators in ATM simulation. It also provides NASA Langley’s ATOL with 
the ability to include MACS capabilities and operator workstations into their simulations. 
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Figure 6: Multi aircraft pilot stations in the AOL. 




Figure 8: One of eight ASTOR flight simulator stations in the AOL. 



E. Air Traffic Management 

MACS has provided considerable air traffic management simulation capabilities for several years now 9,10 . These 
capabilities were designed to simulate future trajectory -based operations and did not attempt to precisely replicate 
existing functions in the field. In support of research on NASA’s ATD-1 project, the flight deck and air traffic 
control simulation capabilities have been fully integrated with TMA, a fielded metering system originally developed 
at NASA Ames. For this integration, the Aeronautical Radar and Data link Simulator (ADRS) (the simulation 
communication network) 
has been extended to 
provide data connections to 
TMA that provide and 
consume the same data 
formats used by the FAA’s 
fielded host, ERAM and 
STARS systems. This 
enables NASA and its 
partners to evaluate its 
near-term ATD-1 TMA 
functional enhancements 
for terminal metering in a 
realistic environment. 

This connectivity also 
enables NASA to include 
TMA into other projects 
focused on trajectory-based 
operations, as well as 
additional TMA features 
and functions. For example, 

Figure 9 shows an arrival 
management station that 


Figure 9: Integrated arrival management station with MACS (left) and TMA 
(right) displays. 
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combines MACS prototype functions with TMA. This station was used for a recent simulation investigating new 
ways to coordinate the intersecting runways at Newark airport 11 . This research will be described in the second half 
of this paper. 

F. Airspace and Complexity Management 

MACS incorporates extensive capabilities to investigate airspace and complexity management research 
questions. These capabilities include powerful tools for complexity assessment, airspace sector combining and de- 
combining, dynamic sector redesign and dynamic reconfiguration. These capabilities have been used in various 
research projects for NASA and the FAA 12 . Figure 10 shows a configuration that was used for an evaluation of the 
Operational Airspace Sectorization Integrated System (OASIS) advisory tool in the AOL 13 



Figure 10: Evaluation of the OASIS advisory tool in AOL with two MACS displays for 
airspace and complexity assessment and OASIS tablet (center). 


G. Manual, Hybrid and Autonomous Air Traffic Operations 

In support of more advanced future concepts, MACS incorporates the capability to conduct air traffic operations 
at different levels of human/automation function allocation, ranging from manual, to hybrid operations, to fully 
autonomous air traffic operations. 

This capability is achieved by utilizing 
NASA technologies for scheduling and 
spacing as well as conflict detection 
and resolution, such as the 
Autoresolver and TSAFE functions 14,15 
for resolving mid- and short-term 
conflicts by the air traffic control 
system, and integrating them with 
digital data communications between 
the ground-based and the airborne 
systems. These functions can be 
utilized manually by air traffic 
controllers or autonomously by the 
system or in a hybrid mode based on 
tolerances that can be selected by the 
experimenter or operator. Using these 
functions, studies on evaluating the Figure 11: Human/autonomy teaming during simulation on 

impact of uncertainties on autonomous automated separation assurance. 
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system functions and on human/autonomy teaming can be conducted. Figure 11 shows an air traffic controller 
display designed for managing largely autonomous air traffic operations. Green aircraft manage their own 
separation; white aircraft are managed by the ground automation. The two aircraft with rectangles around them have 
been taken over manually by the controller to prevent an undesirable conflict resolution from automatically being 
issued by the system. Figure 12 shows controllers monitoring the simulated air traffic system conducting largely 
autonomous operations. 



Figure 12: Air traffic controllers monitoring largely autonomous operations. 


III. Research Activities in the AOL 

Research activities in the AOL are focused on system level evaluations and human/automation integration and 
teaming aspects for advanced operations enabled by new NASA ATM technologies. The activities align with three 
major categories and target epochs: Near-Term Improvements , Initiating Trajectory -Based Operations and 
Increased System Autonomy. 

A.Near-Term Improvements 

When targeting near-term improvements to the NAS, typically the highest level of fidelity is required to properly 
reflect the actual environment, in which new technologies or concepts are to be deployed. By accurately emulating 
fielded systems and integrating new technologies into full scale simulations, the AOL can quickly provide early 
estimates of system-level effects and support design iterations at early stages. This can largely improve the 
probability of a successful introduction of new concepts and technologies. 

1. System-Level Evaluation of ATD-1 Operations 

NASA initiated ATD-1 to demonstrate increased, more consistent use of Performance-Based Navigation (PBN), 
demonstrate an Automatic Dependent Surveillance Sytem - Broadcast-In (ADS-B-In) spacing application, and 
accelerate the transfer of NASA scheduling and spacing technologies for operational deployment. ATD-1 is a multi- 
year collaborative effort between researchers at NASA Ames and Langley Research Centers, the FAA, and industry 
partners to integrate, mature, and operationally demonstrate NASA-developed technologies that have attained a 
sufficient level of maturity to merit in-depth, system-level research in relevant environments 16 . 
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(ASTAR) algorithm 

developed at NASA 
Langley. ASTAR uses 
ADS-B data to provide 
speed commands to flight 
crews, enabling aircraft to 
precisely achieve an 
assigned spacing interval 
behind a target aircraft at a 
specified 4 achieve -by’ 

point. FIM operations are 
expected to reduce 

TRACON controller 

workload and contribute 

additional inter-arrival precision for increasing the proportion of uninterrupted CDOs achievable with high 
throughput. 

Although ATD-1 has a distinct TRACON focus, operations begin when TMA-TM acquires each aircraft while it 
is still in cruise. TMA-TM assigns aircraft a runway, and computes estimated times -of- arrival (ETAs) at the meter 
fix, runway, and at intervening metering points (e.g., TRACON fixes where RNAV routes merge). It then uses the 
ETAs, together with required spacing information, to assign scheduled times-of-arrival (STAs) at each scheduling 
point. When an aircraft reaches a ‘freeze horizon,’ specified at a site-specific distance (e.g., 200 nmi) from the 
TRACON, TMA-TM locks in the aircraft’s STA to provide a stable control target. En-route controllers then begin 
working to ‘precondition’ the aircraft using vectoring or other techniques as necessary to reduce the delay and make 
sure the aircraft is within the speed-control margin. Following preconditioning, en-route controllers reestablish 
aircraft on RNAV routes (e.g., by clearing aircraft direct to the meter fix), and clear aircraft for CDOs. These 
procedures enable flight crews to use their onboard Flight Management Systems (FMSs) to fly efficient profiles 
from cruise until landing (or until a radar vector is required to turn the aircraft to join the approach procedure). 

Once a FIM-equipped aircraft has been preconditioned and established on an RNAV OPD, the controller may 
issue it a FIM clearance. After the crew enters the required parameters into the onboard system (e.g., an Electronic 
Flight Bag (EFB)) and the target aircraft enters ADS-B range, the FIM avionics begin displaying speed commands 
to achieve the required spacing at the achieve -by point. For non-FIM aircraft, TRACON controllers use the CMS 
tools to issue speeds to ensure proper inter-arrival spacing while maintaining CDOs. Under nominal conditions, 
aircraft should cross the meter fix within their speed-control margin for correcting residual schedule errors and 
adjusting for disturbances due to winds or other factors inside the TRACON. Because controllers retain separation 
responsibility, they may find it necessary to interrupt CDOs or suspend FIM operations if they deem small 
adjustments from the nominal speed profile insufficient to maintain safe separation. All of the procedures and 
clearance phraseology for these operations are documented in the ATD-1 Concept of Operations 17 . 

After a systematic evaluation of component technologies in highly coordinated studies, a large scale system-level 
evaluation of ATD-1 operations was conducted in three phases in 2013 and 2014. These simulations, entitled 
Controller Managed Spacing for ATD-l/#5 (CA5) were three large-scale, distributed air and ground simulations 
intended to quantify expected efficiency and capacity gains under realistic operational conditions when using the 
ATD-1 ground-based and airborne technologies. In the first simulation (‘CA-5.1’), controllers worked simulated 
traffic using current-day workstations and control techniques. The second simulation (‘CA-5.2’) introduced the 
TMA-TM and CMS tools, while the third simulation (‘CA-5.3’) added the airborne technologies. All simulations 


Traffic Management Advisor 
with Terminal Metering (TMA-TM) 

Figure 13: ATD-1 Technologies. 


ATD-1 integrates scheduling automation, Controller-Managed Spacing (CMS) tools, and ADS-B-enabled Flight 
Deck-based Interval Management (FIM) avionics (see Figure 13). The scheduling system is the Traffic Management 
Advisor for Terminal Metering (TMA-TM) developed at NASA Ames. TMA-TM represents a significant 
advancement from the currently deployed TMA automation system originally developed as part of NASA’s Center- 
TRACON Automation System (CTAS) 5 . To construct arrival schedules tailored for high-throughput Continuous 
Descent Operations (CDOs), TMA-TM computes trajectory predictions using Area Navigation (RNAV) routes in 
the TRACON, rather than simple transit-time estimates, to generate schedules that are de-conflicted at each 
TRACON merge point. 

FIM capabilities are 
implemented using the 
Airborne Spacing for 
Terminal Arrival Routes 



Flight Deck 
Interval 
Management 
(FIM) 


Controller 

Managed 

Spacing 

(CMS) 
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Figure 14: Airspace for System-Level Evaluation of ATD-1 Technologies. 

used east- and west-flow traffic scenarios derived from recorded Albuquerque Center (ZAB) and Phoenix TRACON 
(P50) traffic that included peak-period arrivals into Phoenix Sky Harbor International Airport (PHX) in different sets 
of historical ZAB truth and forecast winds. Former ZAB and P50 traffic managers operated the Traffic Management 
Advisor to create an efficient arrival schedule. Eight former ZAB controllers used MACS ERAM workstation 
emulations to meter traffic into P50. Former P50 controllers then used MACS STARS workstation emulations with 
CMS tools to manage the aircraft along efficient descent paths according to the schedule. Eight airline pilots flew 
ASTOR desktop flight simulators. Five confederate controllers and eighteen general aviation and corporate pilots 
also participated. Controllers managed approximately seventy-five arrivals per hour-long scenario, along with 
approximately two hundred departures and over-flights. A map of the airspace used during the CA5 simulations is 
shown in Figure 14. 

The CA-5.1 and CA-5.2 system-level simulations suggest that ATD-1 ground-tool operations are viable in a full- 
scale operational environment with controllers who have received only a limited amount of training on the concept 
and CMS tools. While their unfamiliarity appears to have contributed to slightly lower acceptability than would be 
expected from experienced controllers, performance generally improved under operations with ATD-1 ground-tools. 
CMS tool ratings again were highly consistent with those observed in prior research, with controllers finding the slot 
markers most usable and useful. 18 

CA-5.3 afforded examination of all aspects of ATD-1 operations in a realistically complex traffic environment. 
The TMA-TM used runway allocation and an adaptation updated to include all aircraft types which, particularly for 
PHX west-flow operations, typically led to several aircraft scheduled on cross-over routes. Digital and questionnaire 
data were collected for all trials, and are currently being analyzed and correlated with experimenter observations. 
Data are expected to provide detailed information about FIM clearances, FIM in relation to en-route flow 
conditioning, FIM aircraft behavior relative to the slot markers, FIM operations in crossover situations, FIM speed 
commands, FIM spacing performance, TMA-TM schedule performance, CMS tool performance, ATD-1 Minimum 
Operation Performance Standards (MOPS) values, and also pilot and controller training, workload, and 
acceptability. More information is available in 19 . 

The ATD-1 technologies are being transferred to the FAA, with the ground-based technologies representing core 
components of the FAA’s Terminal Sequencing and Spacing (TSS). In order to support the technology transition 
and the FAA’s decision making, the Arrival Metering Precision Study (AMPS), described next, was conducted in 
the AOL in May 2014. 
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Figure 15: Options to format the delay countdown 
timer 



Delay Countdown Timer (DCT) 
+02:00 at Mach .77/280 kts 
+00:50 at 250 kts 


2. Arrival Metering Precision Study (AMPS) 

In May 2014 the Arrival Metering Precisions Study 
(AMPS) was conducted to provide supporting data for 
the introduction of ATD-1 technologies. The 
effectiveness of ATD-1 ground-based technologies and 
TSS will be impacted by the accuracy at which aircraft 
are delivered by Center controllers to the TRACON 
boundary. Arrival metering in the en-route airspace is 
performed using Time-Based Flow Management 
(TBFM) equipment, the fielded version of TMA. When 
activated, this equipment relays each aircraft’s ST A 
and current estimated delay value to the air traffic 
controller’s ERAM workstation. The delay countdown 
timers (DCTs) on the controller workstations can be 
configured with different levels of precision. The 
currently available options are displaying the delay 
rounded to 10’s of seconds, rounded to the next minute, 
or truncated to minutes (see Figure 15). 

The first research objective for AMPS was to assess 
the delivery accuracy at the TRACON boundary for the 
different DCT resolutions. The second question was 
aimed at the use of the controller entered speed intent to 
update the ETA. Prior research 22 has shown that not 
knowing the speed intent of the aircraft has a significant 
impact on the ETA and DCT accuracy, and can 
contribute to reduced efficiency and increased controller 
workload. The FAA’s Ground-based Interval 
Management-Speed (GIM-S) function will utilize speed 
intent in addition to providing speed advisories. While 

AMPS did not use speed advisories, the speed intent was forwarded to the TBFM prototype to be reflected in the 
ETA and DCT (see Figure 16). 

The delay magnitude was varied within each traffic scenario and created by restricting the arrival rate over the 
comer post within TMA. This led to a non-uniform delay distribution with the desired properties, simulating a TSS 
environment without actually 
populating the entire airspace. 

DCT resolution and 4 th line speed 
condition were varied within 
subjects between runs. It is 
expected that the delay resolution 
has a significant impact on the 
delivery accuracy while the use 
of the 4 th line speed may have an 
impact on controller workload 
and flight efficiency. Figure 17 
shows a visualization of the 
study’s design. 

Data for AMPS was collected 
during a simulation of arrival 

traffic into the Northwest corner of Atlanta airspace in May 2014 using realistic traffic samples. The primary 
participants were four air traffic controllers that had retired less than a year before the study. The data analysis is 
currently underway and results are expected to be presented later this year. Airspace and scenarios re-used much of 
the environment created for studying the impact of trajectory prediction uncertainty on controller performance and 
acceptability of automation, described next. 


Figure 16: Use of controller entered speed to update the 
DCT. 
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Figure 17: Factors varied for AMPS. 
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B. Initiating Trajectory-Based Operations (TBO) 

The US and Europe have made four-dimensional (4D) -TBO a primary cornerstone of their air traffic initiatives 
NextGen 20 and SESAR 21 . The key element of the conceptualized TBO is that the airline and the ATM negotiate the 
trajectory that will be followed by an aircraft. This trajectory should satisfy many of the airline preferences and 
include additional constraints, such as controlled times of arrival to improve predictability for the air traffic 
controllers. Initiating TBO is considered a critical step in moving towards NextGen and beyond. Several research 
activities in the AOL help evaluate the effectiveness of particular aspects of TBO. One study investigated the impact 
of trajectory prediction uncertainty on controller performance and acceptability of automation on a more 
fundamental level. Also, to explore the effectiveness of NextGen technologies and TBO within the most challenging 
environment, substantial research activities in the AOL are using the New York area airspace to identify potential 
operational improvements. The third TBO-related research activity in the AOL investigates the integration of arrival 
and departure trajectory planning and scheduling. All three activities are described below. 


7. Impact of Trajectory Prediction Uncertainty on Controller Performance and Acceptability of Automation 

Predictability is a major concern in TBO. Without accurate intent information, a trajectory predictor has to guess 
at the future route, altitude and speed of aircraft. Inaccurate forecasts and models of the environment, aircraft 
performance and flight technical errors cause additional uncertainties that will negatively impact the quality of the 
trajectory predictions. Given the many potential error sources in trajectory-based systems, a fundamental question 
has to be answered when deploying new trajectory-based technologies and moving towards TBO: “How accurate 
must a trajectory prediction be to support successful NextGen TBO concepts? ” 

The overall approach to addressing the problem was to first conduct fast-time simulations to identify and 
quantify the primary and most significant error sources, and then to run a human-in-the-loop (HITL) experiment to 
determine the controller workload and performance under variations of the dominant error sources. 

The HITL presented air traffic controllers with challenging, but manageable, time -based metering problems, 
formulated specific performance targets for the controllers and then varied the errors and uncertainties to see when 
the performance would start degrading. We had postulated that if the automation performance degraded, the 
controllers would have to issue more clearances to correct for the bad “automation advice”. Eventually, they would 
run out of resources and could not issue any more clearances. We expected to see performance targets not being met 
and that we would have found the point at which the automation was not accurate enough anymore. 

The following performance targets were chosen: 

a) Encounter No Loss of Separation (LOS) events 

b) Deliver aircraft to the meter fix as accurately as possible, at least within 25 seconds of their STA 


The following errors had been identified to be realistic error 
sources and have the biggest impact on the automation performance: 

• Wind forecast errors 

• Aircraft performance errors 

• Flight technical errors 

Figure 18 shows the error conditions tested in the study. They 
included a no error condition, realistic errors (e.g. 10 kts wind error, 
5% error in predicted descent performance) as well as largely 
exaggerated error conditions (e.g. 40 kts wind error, 25 % descent 
performance). 
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Figure 18: Tested combinations of 
wind forecast error and aircraft 
performance error. 


Other factors that play a role in metering situations are controller technique and task difficulty. In order to gather 
a more comprehensive cross section of metering situations we decided to employ at least two different controller 
teams with potentially different techniques and to run the error conditions under varying task difficulties. More 
details about the experimental design and detailed results are presented in 22 . 

The most significant finding from the study is depicted in Figure 19. For this analysis, when aircraft crossed the 
meter-fix, aircraft were classified into one of three groups: on time, early, and late. An aircraft was considered to 
arrive at the meter-fix on time if it arrived within 25 seconds of its STA. Aircraft arriving more than 25 seconds 
early relative to their STA were considered ‘early’, and those arriving more than 25 seconds late relative to their 
STA were considered Tate’. The formula used in this analysis was simply STA - Actual Time of Arrival (AT A), 
with positive values indicating early arrivals, and negative values indicating late arrivals. Overall performance was 
high; 578 of 598 aircraft (97%) were delivered on time. In comparisons by error condition and traffic scenario, 
schedule conformance at the meter-fix was always at a 94% success rate or higher. Additionally, the raw STA-ATA 
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values were tested for statistical significance across error condition and traffic scenario. Significant differences in 
data were not found in either comparison, indicating that the controllers achieved similarly high performance 
regardless of error condition or traffic scenario. 



Figure 19: Percentage of aircraft delivered to the meter-fix either early, late, or on time, when 
compared across error conditions and traffic scenarios. 


The results suggest that even in the largest trajectory prediction error conditions, controllers were able to learn 
how to compensate for the errors and adapt their interaction with the tools to deliver arrival aircraft on time and not 
exceed workload limits. In fact, the error conditions exhibited minimal impact on performance; rather, it was other 
factors, such as traffic scenario and tool availability, that had measurable impact on system performance. It is 
believed that the size of the errors managed by the controllers in this manual condition may prove difficult for a 
more automated system built to issue corrective updates automatically. This suggests the opportunity to enhance 
research on more far-term concepts and to incorporate corrective learning; much like the controllers did naturally in 
the simulation. In the presence of these errors, it is also likely that the transition from a controller-in-the-loop 
paradigm to a more autonomous system may be problematic without significant investigation and improvement to 
the learning abilities and calibration of the automation and the human/automation teaming concept. 

2. New York Operational Improvements 

The New York area airspace is notoriously complex and demanding, and is responsible for a large portion of the 
delays and congestion in the United States. A new research effort was initiated to examine the chronic problems 
associated with New York, and to develop integrated NextGen concepts that leveraged existing NASA and other 
NextGen technologies to reduce delays, increase throughput, and/or improve trajectory efficiencies in this region. 

An initial HITL simulation in the AOL addressing the New York airspace explored the feasibility of a NextGen 
TBO solution to address airspace and airport capacity limitations in and around the New York Metroplex 11 . A 
concept, tools, and operational procedures for improving flight efficiency and runway throughput for EWR arrivals 
were developed to create a precise, dependent-runway schedule between two intersecting runways. A week-long 
study, conducted in August 2013, explored the feasibility of a new Optimal Profile Descent (OPD) arrival into the 
airspace as well as a novel application of the TMA-TM arrival scheduling tool to coordinate high volume arrival 
traffic to intersecting runways. In the simulation, four en route sector controllers and four TRACON controllers 
managed traffic inbound to EWR’s primary runway, EWR22L, and its intersecting overflow runway, EWR11. 
TMA-TM was used to generate independent arrival schedules for each runway, while a traffic management 
coordinator participant adjusted the arrival schedule so that each EWR11 followed a EWR22L aircraft. CMS tools 
were also provided to assist the TRACON controllers in managing the arrivals descending on OPDs. 
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Figure 20: Trajectories from 2011/06/27 operational data (left), Baseline condition (middle) and Futures 
condition (right). 

The study compared the throughput and safety of an experimental condition labelled Futures using the TMA-TM 
tools with dependent-runway scheduling, with a Baseline condition that did not. Figure 20 shows the lateral 
trajectories of the arrivals on OPDs in the Baseline and Futures conditions (5 hours of Baseline and Futures 
condition runs, with 190 and 198 simulation trajectories, respectively), compared to 24 continuous hours of actual 
operational data for arrivals from ZDC for EWR22L or EWR11 on a clear weather day (June 27, 2011, 316 aircraft 
trajectories). The figure illustrates that the OPD RNAV trajectories used in the study resulted in fewer lateral path 
deviations than the trajectories from the operational data. 

The results suggested that the efficient RNAV descent procedures were compatible with converging-runway 
operations using the modified TMA-TM and the controller tools. Further, the combined use of the tools with the 
new OPDs produced a peak arrival rate of 67 aircraft per hour using instrument flight rules (IFR), exceeding the 
current maximum arrival rate at EWR of 52 per hour under visual flight rules (VFR). In the Futures condition, the 
controllers were able to deliver arrival aircraft precisely on OPDs at a high throughput rate for both EWR22L and 
EWR11 runways with less vectoring, fewer lateral path deviations, and a lower workload. More importantly, a 
coordinated schedule across the two intersecting runways at a high-throughput rate was successfully managed with 
minimal coordination using the TMA-TM tools. A coordinated schedule allowed the controllers to deliver their 
aircraft to the slot markers with the assurance that they would result in conflict-free delivery at the converging 
runways. The schedule and the slot markers were created taking winds in the account, and therefore the controllers 
could deliver the aircraft safely even during high wind conditions. In the Baseline condition, the controller needed 
to make last minute path adjustments on the EWR11 arrivals in order to pair them with the leading EWR22L 
arrivals, resulting in more lateral path deviations. 

In terms of safety, the "go-around" 
violation data suggested that only the Futures 
condition could safely increase throughput. 
The go-around violations were logged 
whenever the arrivals on EWR1 1 and 
EWR22L were spaced too closely to each 
other during landing. A go-around violation 
was defined as occurring if the aircraft 
landing on EWR1 1 was at its threshold and 
the aircraft landing on EWR22L had not yet 
crossed the runway intersection and was less 
than 1.5 miles out. Figure 21 shows all of the 
go-around violations that occurred in the 
study. There were three go-around violation 
cases observed in the Futures condition, but 
they were borderline cases that, as identified 
during a follow-up discussion with a subject- 
matter expert from EWR tower, in today’s 
operations would not be considered a 
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Figure 21: Go-Around Violations. 
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violation. Therefore, the go-around violations with safety implications occurred only in the Baseline condition 
(circled in Figure 21). 

A follow-up benefit analysis was performed to evaluate the annualized benefit of the increased traffic once the 
realistic wind and traffic demand were taken into account. 23 The analysis examined the meteorological conditions 
suitable for using the 11-22L landing runway configuration, and estimated the actual resulting potential for 
increasing arrival throughput. Aviation System Performance Metrics (ASPM) data were used to determine the traffic 
demand, runway configurations, meteorological conditions, and wind magnitudes/directions during 2007, 2009 and 
2012. The annualized benefit analyses consisted of two parts: (1) identifying the availability of using runway 11 in 
addition to runway 22L under favorable wind and meteorological conditions beyond the current level of its usage, 
and (2) applying the increased throughput identified by the simulation study to the 1 1 -22L arrival configuration that 
can be used across an entire year. 

Using fairly conservative assumptions about the meteorological conditions, runway usage, and peak demand 
characteristics, the analysis showed that the 1 1 -22L runway configuration could have been used more than twice as 
often during high-demand periods (i.e., when arrival demand exceeded throughput), which in turn could have either 
reduced delays to the impacted aircraft by approximately 15 minutes per aircraft, or increased the overall throughput 
by approximately 8% - 11% per year. This new operation has the potential to deliver even greater benefits if it were 
extended to include other runway configurations and demand conditions. The results suggest that a large 
throughput/delay benefit can be gained from a modest extension of the TMA-TM and make a significant impact in 
the New York area. 

3. Sharing of Airspace Resources (SOAR) 

The AOL’s research group focused on Sharing of 
Airspace Resources (SOAR) is working to develop tools and 
procedures for conducting Metroplex operations with 
arriving and departing aircraft flows that share the same 
airspace. The concept of shared airspace in TRACONs 
allows more efficient routes by sharing a common airspace 
across different arrival and departure flows. Aircraft from 
one flow could use the same airspace as another flow, 
provided there are available gaps in the latter flow; this 
interaction of departure and arrival flow require timely 
coordination of traffic by controllers. Modeling studies have 
shown that the hybrid use of spatial and temporal spacing 
supports more efficient routes. 

A first evaluation was conducted in the AOL in 2012 
using trajectory-based tools to coordinate departures leaving San Jose airport through gaps in the San Francisco and 
Oakland arrival stream (see Figure 22). The simulation assessed the efficiency and safety of 96 departures, during 
which the SJC tower had a tool to identify departures that could fly through predicted gaps in the arrival flow. 
Otherwise, a safe, but less efficient, route was used to keep the departures underneath the arrival flows. 
Additionally, the arrival controller was able to control the SJC departures right after take-off using a point-out 
coordination procedure. The simulation manipulated the accuracy of departure time (accurate vs. inaccurate) as well 
as which sector took control of the departures after take-off (departure vs. arrival sector) in a 2x2 full factorial 
design. Results show that coordination time decreased and climb efficiency increased when the arrival sector 
controlled the aircraft right after take-off. Also, climb efficiency increased when the departure times were more 
accurate. Coordination was shown to be a critical component of tactical operations in shared airspace. Although 
workload, coordination, and safety were judged by controllers as acceptable in the simulation, improved tools and 
coordination procedures seemed warranted 24 . 



Figure 22: Arrival Departure problem addressed 
in SOAR 1 simulation in 2012. 
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Therefore, a second 
research phase situated in 
the New York airspace is 
currently underway. In 
this simulation, more 
emphasis is placed on 
developing tools and 
procedures for combining 
temporal and spatial 
information in the 
complex Metroplex 

environment. These tool 
prototypes have been 
developed and 

implemented in MACS 
and include Multi-Route 
and Route-Crossing tools 
and capabilities for 
departure release 

coordination. Figure 23 
depicts the Multi-Route Figure 23: Tools developed for the SOAR project, 
and Rout-Crossing tool 

prototypes in an environment in which EWR arrivals share airspace with La Guardia (LG A) arrivals. 



ft 


H 1B r 


AAL9B6>- J L 

-<AAL906 

DAL333-TWO>- ftft 

DAL 333 -TWO 

AAL334>- | 

-<AAL334 

r s r 

AAL755>- I 

-<AAL 7 55 

0 

FP ETA RABBA FP ETA 


An exampleofthe use of the Multi-Route, 
the Route CrossingTool and the Arrival & 
Departure Timeline to plan a departure and 
select a route for EWR departure DAL535 
that is estimatedto cross the LGA arrival 
flow at MOFT27.0 nm behind LGAarrival 
AAL334 and 9.0 nm in front of AAL334. 


C. Increased System Autonomy 

TBO can be considered a main enabler of many NextGen and SESAR functions, and is also an important 
stepping stone towards a true transformation of the airspace system beyond NextGen. However, given the much- 
documented limitations of the current system, there also needs to be a fundamental paradigm shift in the allocation 
of functions between humans and automation to truly transform the airspace system towards increased mobility, safe 
and efficient growth in global operations, and enabling many of the new vehicles and operations that will be coming 
on-line over the next decades 25 . Moving beyond the human-operated manual air traffic management system that we 
have today will require highly capable systems that can conduct many operations autonomously as well as effective 
human/autonomy teaming strategies that ensure safe and efficient operations. NASA has conducted research on air 
traffic control operations under higher levels of automation and autonomy for over more than a decade. In close 
cooperation with other researchers at NASA Ames and Langley, the AOL integrated many concepts and algorithms 
into the envisioned air traffic operational environments and conducted a large body of fundamental human/systems 
integration research that has provided valuable insights into limitations and opportunities of different air traffic 
paradigms. The subsequent section summarizes some of this research. More detailed information is available 

• n 25,26,27,28 


7. Air Traffic Control Operations under Higher Levels of Automation and Autonomy 

In 2008 the AOL started a series of studies focused on safely increasing capacity through new ways of 
human/automation collaboration for separation assurance. The studies were conducted with the over-arching goal of 
determining whether separation assurance automation can be integrated into air traffic control operations in an 
acceptable and safe manner. These studies investigated a range of issues including the proper levels of automation 
for given capacity targets, off-nominal operations from both air and ground perspectives, and sustained near-full- 
mission operations with many tasks allocated to the automation in the presence of convective weather and 
scheduling constraints. 

Overall, it was found that, if properly integrated, advanced air traffic control automation has the potential to 
solve the envisioned airspace capacity problem. The automation was largely effective and robust, and an acceptable 
function allocation strategy between controllers and automation began to develop: allocating routine conflict 

avoidance to the automation, unusual situations to the controller, and providing information about short-term 
conflicts that gives the controller a large enough window of opportunity during which they can intervene with a 
solution. The studies started to identify the proper balance between the roles of humans and automation to maintain 
a consistent and appropriate level of engagement for the controllers. Controllers were comfortable with the 
automation dealing with several routine tasks without their involvement, but wanted decision-making authority and 
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support in maintaining an overall awareness. The three studies have also served to shed light on specific tasks where 
the allocation of function was less clear, such as handling short-term problems. Feedback and results also showed 
that further technological development is necessary to improve trajectory prediction and conflict detection accuracy. 
The need for further procedural development to govern controller/automation teaming and air/ground interactions 
was also highlighted 25 . 

Following up on the earlier findings a 2012 human-in-the-loop air traffic control simulation investigated a 
gradual paradigm- shift in the allocation of functions between operators and automation. Air traffic controllers 
staffed five adjacent high-altitude en route sectors and, during the course of a two-week experiment, worked traffic 
under different function-allocation approaches aligned with four increasingly mature NextGen operational 
environments (see Figure 24). The traffic was primarily ground -managed with a small percentage of aircraft 
conducting airborne self-separation operations. The four NextGen ‘time-frames’ ranged from near current-day 
operations to nearly fully-automated control in which the ground system’s automation was responsible for detecting 
conflicts, issuing strategic and tactical resolutions, and alerting the controller to exceptional circumstances. The 
study provided a unique opportunity to investigate transitional stages. 




The first stage, “Current Day”, was designed to provide data approximating current day operations with the 
addition of ADS-B out surveillance data. The traffic levels were selected to be representative of current day peak 
traffic levels with a Monitor Alert Parameter (MAP) value of 18 aircraft per sector. 

The second stage, labeled “Minimum NextGen”, introduced limited data communication between the ground - 
side and 25% of the simulated aircraft. This data communication enabled an automatic transfer of communication of 
aircraft from one sector to the next. This eased the controller workload in handling those aircraft. It was expected 
that controllers could potentially ignore self-separating aircraft, because they had no routine duties with regard to 
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them. This stage also introduced more decision support capabilities for the controllers, none of which were 
integrated with data comm. So, all control instructions still had to be communicated via voice. It was hoped that the 
new technologies could enable a capacity increase of 20%, and the MAP value was set to 22 aircraft per sector for 
the “Minimum NextGen”. 

In the third stage, entitled “Moderate NextGen”, the controller planning tools and the flight management systems 
on-board the aircraft were integrated with data comm., and 50% of the aircraft were assumed data comm, equipped. 
Controllers were able to issue trajectory change instructions to equipped aircraft via data comm. Based upon earlier 
research, it was hypothesized that this environment could enable a capacity increase of 50% over the Baseline and 
therefore the MAP value was set to 27 for this stage. 

In the final NextGen stage, referred to as “Maximum NextGen”, the automation provided separation assurance 
autonomously and all aircraft were data comm, equipped. When conflicts were detected, automation computed 
trajectory-based resolutions and issued those directly to the flight deck, as long as the computed resolutions did not 
violate preset tolerances. Otherwise, the conflict was flagged to the controller for resolution. Prior research had 
indicated the scalability of this approach, and therefore the traffic levels were selected at 100% over the Baseline 
with a MAP value of 36 aircraft per sector. 



Figure 25: Peak aircraft count and workload rating for 


Figure 25 contrasts the measured aircraft count to subjective controller workload ratings and shows that the 
aircraft count increased from the Baseline to the Minimum, the Moderate and the Maximum condition, whereas the 
workload stayed constant for Baseline, Minimum and Moderate conditions. Only the Maximum condition showed a 
significantly lower workload. The maximum highly automated condition outperformed the others also the other 
metrics, such as flight path efficiency while not compromising safety over the baseline. 26,27 28 . 

Somewhat unexpectedly it was also found that the maximum condition achieved the highest situation awareness 
and acceptability ratings (as shown in Figure 26). 



Figure 26: Situation awareness and acceptability ratings 


Conversely, the moderate condition was the most problematic, exhibiting more separation violations than the 
others (not shown here) and the lowest acceptability ratings. This indicates that simply adding automated functions 
without changing the operational paradigm may be problematic. The results provide much support for pursuing 


18 

American Institute of Aeronautics and Astronautics 


concepts of increased autonomy and developing human/autonomy teaming strategies that can effectively utilize the 
considerable benefits of advanced air traffic control automation. 

IV. Conclusion 

The Airspace Operations Laboratory at NASA Ames Research Center has developed powerful capabilities for 
efficiently conducting research on the future air transportation system. Several additional NASA technologies have 
been integrated with the AOLs primary simulation capabilities where appropriate to conduct system level 
evaluations of near-term technologies that are being transitioned to the field. The AOL’s rapid prototyping and 
flexible simulation capabilities have proven a highly effective environment to progress the initiation of trajectory- 
based operations and support the mid-term implementation on NextGen. Fundamental questions about accuracy 
requirements are addressed as well as real world problem on how to improve operations in some of the most 
complex airspaces in the US. Looking beyond NextGen, the AOL has started exploring hybrid human/automation 
control strategies and largely autonomous operations in the air traffic control domain. Initial results indicate largely 
improved capacity, low operator workload, good situation awareness and acceptability for controllers teaming with 
autonomous air traffic systems. While much research and development needs to be conducted to make such concepts 
a reality, these approaches have the potential to truly transform the airspace system towards increased mobility, safe 
and efficient growth in global operations and enabling many of the new vehicles and operations that are expected 
over the next decades. 
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